From Common to Special: When Multi-Attribute Learning Meets Personalized Opinions

Visual attributes, which refer to human-labeled semantic annotations, have gained increasing popularity in a wide range of real world applications. Generally, the existing attribute learning methods fall into two categories: one focuses on learning user-specific labels separately for different attributes, while the other one focuses on learning crowd-sourced global labels jointly for multiple attributes. However, both categories ignore the joint effect of the two mentioned factors: the personal diversity with respect to the global consensus; and the intrinsic correlation among multiple attributes. To overcome this challenge, we propose a novel model to learn user-specific predictors across multiple attributes. In our proposed model, the diversity of personalized opinions and the intrinsic relationship among multiple attributes are unified in a common-to-special manner. To this end, we adopt a three-component decomposition. Specifically, our model integrates a common cognition factor, an attribute-specific bias factor and a user-specific bias factor. Meanwhile Lasso and group Lasso penalties are adopted to leverage efficient feature selection. Furthermore, theoretical analysis is conducted to show that our proposed method could reach reasonable performance. Eventually, the empirical study carried out in this paper demonstrates the effectiveness of our proposed method. Introduction Visual attributes, which describe human labeled properties (like open, fashionable) for a given image, have shown its great potential as a mid-level semantic cue to enhance a variety of applications including face verification (Song, Tan, and Chen 2014), person re-identification(Su et al. 2016; Su et al. 2017), and zero-shot learning (Ji et al. 2017; Wang et al. 2017; Zhang and Saligrama 2016), etc. Generally speaking, there are two types of attributes: i) binary attributes express whether a property is absent or present in a given image (like A is/is not open); ii) relative attributes show the strength of an attribute conveyed in one image with respect to another image (like A is more/similarly/less open than B) (Parikh and Grauman 2011). On one hand, the attribute predictors are often trained with the crowd-sourced global labels. The justification of such an approach is that there is only one unique ground truth ∗The corresponding author. Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: Illustration of the three-component decomposition. Here a1, a2 and a3 are the three mentioned attributes: high heeled, open and feminine. We assume that, for each attribute, there are two annotators who labeled the corresponding images. Note that we extend θ and P to match the size of U . and the majority of annotators have to recognize this “correct answer” simultaneously. However, different annotators might very well have distinct preferences, such that participants of the crowdsourced experiments might vote under different criteria or conditions. It might be misleading to merely look at a global consensus while ignoring personal diversity. On the other hand, practical visual applications often involve simultaneously learning multiple attributes together. In such a case, different attributes may intrinsically share some common patterns. One reason is that these attributes convey similar semantic meaning. Another reason is that they use common subsets of low level image features. In that sense, training multiple attribute predictors independently might not be an appropriate protocol. Based on the discussion above, our goal is to solve two problems simultaneously in this paper: 1) learning userspecific attributes, 2) learning multiple attributes together with their shared information. For 1), (Kovashka and Grauman 2013) regard userspecific attribute learning as an adaption process. In this work, a generate model is first trained based on a large pool of crowd-sourced labels. Then a small user-specific dataset is employed to adapt the generic model to user-specific predictors. Meanwhile, (Kovashka and Grauman 2015) argue that one attribute may fit to different shades (interpretations) ar X iv :1 71 1. 06 86 7v 1 [ cs .L G ] 1 8 N ov 2 01 7 for different groups of persons. Correspondingly, the authors proposed an automatic shade discovery method to leverage group-wise user-specific attributes. Though these existing works are designed to deal with user-specific attributes, they neglect the mutual interactions between different attributes. Multi-task learning framework is well known as a standard solution for 2). Recently, many efforts have been made to improve multi-task learning. (Ando and Zhang 2005) proposed an alternating structure optimization algorithm to decompose the predictive model of each task into two components: the task-specific feature mapping and task-shared feature mapping. For robust multi-task learning, (Chen, Zhou, and Ye 2011) proposed a corresponding method with a lowrank structure and a column-wise sparse structure. In (Gong, Ye, and Zhang 2012), the low rank structure proposed in (Chen, Zhou, and Ye 2011) was replaced by a row-wise sparse structure to leverage selection of a common subset of features. Since the tasks from the same group are closer to each other than those from a different group, (Zhou, Chen, and Ye 2011) proposed a clustering based multi-task learning framework. Motivated by the fact that the tasks should be related in terms of subsets of features, (Xu et al. 2015) proposed a novel multi-task learning method via task-feature co-clustering. As for applications, (Huang et al. 2014) proposed a robust dynamic multi-task method for trajectory regression. There are also some existing works that focus on applying multi-task frameworks to attribute learning. One typical way to do this is to extend the existed multi-task algorithms to match attribute learning. For instance, in (Chen, Zhang, and Li 2014) the model proposed in (Gong, Ye, and Zhang 2012) was generalized to learn multiple relative attributes with their shared information. Meanwhile, some works employ deep learning methods to solve this problem by partially sharing the learned weights among different attributes. (Ehrlich et al. 2016) proposed a multi-task restricted boltzman machine so as to learn a shared feature representation for multiple facial attribute learning. (Hsieh, Hsu, and Chen 2017) incorporates identity and human attributes in learning discriminative face representations through a multi-task method. A deep multi-task learning approach was proposed in (Han et al. 2017) to jointly estimate multiple heterogeneous attributes from a single face image. (Hand and Chellappa 2017) also proposed a multi-task deep convolutional neural network with an auxiliary network at the top to capture attribute relationships. Though these works have successfully improved attribute learning with multi-task models, as was mentioned previously in this section, they all employ crowdsourced labels to train attribute predictors and ignore the disagreement among users. Note that, except learning global labels for multiple attributes, multi-task frameworks are also suitable for learning user specific attributes where global patterns are necessary for capturing the public opinion, and task-specific patterns are indispensable as well for capturing user bias toward that public opinion. With such belief in mind, different with most of the previous works which partially met the requirement of our goal, we propose a hierarchical multi-task framework where task relationships are modeled on both the attribute level and the user level. The main contributions of this paper are two-fold: • To match the hierarchical nature of the underlying problem, we propose a common-to-special decomposition of the model weights, which captures the general cognition patten, attribute level bias and user specific bias, respectively. An optimization method is established based on the accelerated proximal gradient method. • Theoretical analysis is performed in this paper. The corresponding results show that our proposed algorithm could attain reasonable performance. Methodology In this section, we’ll present an attribute learning method to learn user specific labels across multiple attributes. We first introduce the notations used in this paper. Secondly, we propose our model formulation, which includes a common-tospecial decomposition of the model weights and the corresponding objective function. Thirdly, we introduce our optimization method based on the accelerated proximal gradient method. Finally, the theoretical analysis is carried out to show the performance bound of our method. Notations In this paper, scalars, vectors, and matrices are denoted as lowercase letters (a), bold lower case letters (a), and bold upper case letters (A). Xk denotes the kth row of X . xij denotes the (i, j) entry of a matrix X . P(·) denotes a probability measure. [a] denotes the set :{1, 2, · · · , a}. Given an index set I, AI denotes a matrix that contains all the corresponding rows of A, while aI represents the vector that contains the corresponding elements of vector a. ‖·‖p denotes the `p norm : ‖x‖p = ( ∑